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Abstract. We show that a recent identity of Beck-Gessel-Lee— Savage on 
the generating function of symmetrically contrained compositions of integers 
generalizes naturally to a family of convex polyhedral cones that are invariant 
under the action of a finite reflection group. We obtain general expressions for 
the multivariate generating functions of such cones, and work out the specific 
cases of a symmetry group of type A (previously known) and types B and D 
(new). We obtain several applications of the special cases in type B, including 
identities involving permutation statistics and lecture hall partitions. 



1. Introduction 

Motivated by the "constrained compositions" introduced by Andrews-Paule- 
Riese p], Beck-Gessel-Lee-Savage [2] enumerated symmetrically constrained 
compositions, i.e., compositions of an integer M into n nonnegative parts 

M = Ai + A 2 H h A n , 

where the sequence A := (Ai, A2, . . . , A n ) satisfies the symmetric system of linear 
inequalities 

a>lK(i) + a 2 A 7r(2 ) H h a„A 7r(n) > for all n G S n . 

Specifically, [2j discusses various approaches to compute, for a fixed set of parame- 
ters 01, a%, ■ ■ . , a n , the generating functions 

F{ Zl ,z 2 ,...,z n ) :=Y J ^ 1 z^---z^ 



and 



F{q) :=F(q,q,...,q)=J2q 



Ai + A 2 H hA„ 



A 

where both sums extend over all symmetrically constrained compositions A. One 
viewpoint of [2] is geometric: The compositions (Ai, A2, . . . , A„) are interpreted as 
integer lattice points in the cone 

(1) {x € R" I Vct g W : (ax, a) > 0} , 

where W is the image of the permutation representation of £„, a = (ai, . . . ,a n ), 
and ( , ) is the standard inner product on R". This viewpoint together with 
permutation statistics of S n gave rise to explicit (and in some instances surprising) 
generating function formulas. 

Our goal is to generalize the results in [2] to cones of the form (JT]) where W is 
another reflection group. In addition to obtaining general multivariate generating 
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function identities, we obtain several applications of these results for hyperocta- 
hedral groups. These applications are similar in spirit to the applications in the 
symmetric-group case found in [2J. 

The outline of our paper is as follows. The general setup for our approach is 
discussed in the next section, which also contains our central result, IThcorcm 2.81 
Section [3] illustrates our approach by re-deriving the main result in [2] . Sections |4] 
and [5] consider cones constrained by reflection groups of type B and Z?, respec- 
tively. Further, Section 0] contains applications obtained through specializing our 
generating functions in the type-B case. 
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2. General theory 

Our goal in this section is to study integer points in cones that are constrained 
by the orbit of a single linear constraint under an appropriate group action on real 
space. This goal is realized in lTheorem 2.81 where the multivariate generating func- 
tion encoding the integer points in such a cone is expressed as a sum of simpler 
generating functions. ITheorem 2.81 is an algebraic consequence of a geometric tri- 
angulation of the symmetric cone, which we obtain in lLcmma 2.51 proposition 2.6| 
makes the triangulation disjoint by using combinatorics of Coxeter groups as a 
tiebreaker for the walls separating the maximal cones in the triangulation. This is 
critical for our subsequent applications. 

2.1. Almost irreducible finite reflection groups, Coxeter groups, and de- 
scents. In the following, we will consider finite reflection groups (see, e.g., [9] for 
background) that act on spaces in a restricted fashion. Namely, a finite reflection 
group W C O(V) acting on a Euclidean vector space V is called almost irre- 
ducible if V decomposes into ^-invariant subspaces V = V\ © V2 such that W acts 
irreducibly and nontrivially on V\ and trivially on V2, and that V2 is 1-dimensional. 

Example 2.1. S n acts almost irreducibly on R™ by permutation of the components. 
The irreducible summand consists of all vectors with component sum 0, and the 
trivial summand consists of all vectors with equal components. This is the case 
considered in [2J. 

Example 2.2. Let V\ be a Euclidean vector space and W C O(Vi) a nontrivial 
irreducible reflection group, i.e., a nontrivial reflection group such that V\ does not 
contain any nontrivial proper VF-invariant subspaces. Let W act trivially on R and 
set V = Vx © R. Then W acts almost irreducibly on V. 
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A Coxeter group of rank r is a group admitting a presentation with generators 
si, . . . , s r and relations (sjSk) mjk — 1 for rrijk G {1, 2, 3, . . .} U {oo} subject to the 
conditions that rrijk — nikj and rrijk = 1 < ^=> j = k. Here, a value of rrijk = oo is 
to be understood as the absence of the corresponding relation. Such generators are 
called simple generators. For each Coxeter group considered, we will suppose 
that simple generators have been fixed once and for all. We refer the reader to [4] 
or [9] for further information about Coxeter groups and their relation to reflection 
groups. 

The length 1(a) of an element a £ W of a Coxeter group W is the smallest 
integer such that there is a decomposition a — Sj 1 ■ ■ ■ Sj t .. of a as a product of 1(a) 
not necessarily distinct simple generators. For any a € W , the right descent set 
of a is 

(2) D r (a):={j€{l,...,r}\l(a Sj )<l(a)}. 

Remark 2.3. Propositions 13.11 14. 1[ and 15.11 review the connection between the 
definition of descent given here and definitions of descent for Coxeter groups of 
types A, B, and D in terms of the one-line notation. 

Recall that if W is a finite reflection group, it is automatically a Coxeter group. 
Simple generators can be found as follows. Let H be the union of all reflection 
hyperplanes for W; denote by F the closure of a connected component in V \ H. 
It is immediate that F is a convex polyhedral cone. Let Hi, ... , H r be the facet 
hyperplanes of F and let Si be the reflection at Hi. Then Si,...,s r are simple 
generators of the Coxeter group W. See [5, V.3.2, Th. 1] for the proof of these 
statements. 

A subset F C V is a fundamental domain for W if F is the closure of an 
open set and each PF-orbit intersects F in exactly one point. By j9[ Section 1.12], 
every such F is polyhedral, and is bounded by hyperplanes fixed by a set of sim- 
ple reflections in W. Through the rest of this paper, when given a set of simple 
generators s%, . . . , s r of a reflection group W, we denote by F a fixed fundamental 
domain with bounding hyperplanes corresponding to s%, . . . , s r . 

2.2. Triangulations of monoconditional cones. Denote the value of a linear 
form (p G V* on a vector x £ V by (x, ip). Let W C 0(V) be an almost irreducible 
reflection group. A symmetric cone C C V is a convex polyhedral cone that 
is VF-invariant. A symmetric cone is called monoconditional if there is a linear 
form ip G V*, such that V\, Vi <£ ker(y>) and 

(3) C = {x e V | Vct e W : {ax, ip) > 0} . 
This generalizes ([1]). 

Example 2.4. The positive orthant R^o i s a monoconditional symmetric cone for 
the almost irreducible action of S n on R n by permutation of the components. A 
possible linear form defining it is the projection on the first component. 

Lemma 2.5. Let W C O(V) be an almost irreducible reflection group. Let C C V 
be a monoconditional symmetric cone. Let F C V be a fundamental domain for 
the action of W on V . Then the cone C + :— C n F is simplicial. In particular, C 
admits the triangulation 

C= (J aC+. 
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Figure 1 . Notation used in the proof of ILemma 2.51 The cyclic 
group of order 2 acts almost irreducibly on R 2 by sign change in 
the first component. 

Proof. Some of the notation used in this proof is shown in |Figurc l| for convenience. 
Let (p G V* be a linear form defining C as in (|3]). Let x v G V such that (x, <p) = 
— {x, x v ) for all x G V. Let Xo be the unique element of Wx v R F . Then 

C + = {x £ F \ V a e W : (ax,x ) < 0} . 

Let V = V\ © V2 be the decomposition of V into the irreducible and trivial compo- 
nent. Let xq = x' + x'q with x' e V± and x' ' G V2. By definition V2 ^ ker ((/?), and 
so x v $V\, thus xo $V%, and hence x ' 7^ 0. As (p is only determined up to multi- 
plication by a positive scalar, suppose without loss of generality that (x ',Xq) = 1. 
Let 

P+ = {x e F n Vi I Vcr G W : (ax, x' ) < 1} . 

Consider the reflections at the facet hyperplanes of F as simple generators of W . 
Let I denote the corresponding length function. Let x G F n V\ and <r G W. Let H 
be a facet hyperplane of aF, such that l(so-) > 1(a) for the reflection s at ff. We 
claim that in this situation 

(4) (s<tx,x' ) < {ax,x' ) . 

Indeed, consider the decomposition V\ = (H n V\ ) © . According to this decom- 
position, write x' = vo + wq and ax = v± + Wi; then sax = v\ — w±. We have 
(crx, a; ) = (t>i, «o) + wo) and (sex, x ) = (vi, vo) — (wi, wo)- Hence 

(5) (scrx, a; ) = (ax, x' ) -2(w 1 ,w ). 

Generally, if t G W, then 1(t) equals the number of reflection hyperplanes between 
F and tF. As l(sax) > l(ax), this implies that xo and ax lie on the same side of 
H. Hence (wi.wo) > 0, and so ([5]) implies the claim (j4j). 
By induction on Z(c), (j4|) implies that 

P+ = {xeFnl/i I (x,x ) < 1}. 
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The cone C+ is the cone over P — x'q, thus C+ = {x G F \ (x,xo) < 0}. The 
cone F DVi is a fundamental domain for the action of W on Vi. Hence F D Vi is 
simplicial. We have F = (F n Vi) + V 2 , and so dim(F ) = dim(F n Vi) + 1. The 
cone C+ is defined in F by the single additional inequality (x,xq) < 0, thus C + is 
simplicial. □ 

Consider the situation of lLcmma 2.51 Choose an order Hi, . . . , H n _i of the facet 
hyperplanes of F with the corresponding simple reflections s\, . . . , s n -\. For any 
subset J c {1, . . . , n — 1}, let 

Cj : C (J Hj . 

je./ 

For example, Cg = C+. If J ^ some of the facets of C+ are removed. 
Proposition 2.6. /n i/ie situation o f\Lemma 2.5\ C decomposes as a disjoint union 

C = U a ^D^o) ■ 

Proof. For x £ C, let 

WO) := {ct G V7 | as € o-C+}. 

Bv lLemma 2.5i C = Uo-ew a an( i so ^ ne se ^ W(x) is nonempty. It contains a 
unique element of minimal length [9, §1.10], denoted by a x . 

Assume that &~ 1 x G Hj for some j € D r (a x ). Then s^-cr" a; = a~ x, and so 
.x = a x Sja~ x. As cr ;! r 1 a; € C+, this implies that a x Sj 6 W(a;). On the other hand 
l(a x Sj) < l(a x ), a contradiction. Hence a x Y x ffj for all j G D r (cr x ). Hence 
a^x e C Dt ^ x ), and so x G a x C Dt ^y This proves that C = \J aeW a C Dt (a)- 

To prove disjointness, let x £ aCr) T (a) f° r some cr € IF. We have to show that 
er = a x . Clearly a £ IF(x). It remains to show that cr has minimal length in W(x). 
Assume that a has not minimal length in W(x). Then there is j £ {1, . . . , n — 1} 
such that Z(ctSj) < 1(a) and crsj G W(x) [H §1.10]. From a G W(a;) we conclude 
that (T _1 a; G C+ and from asi G VF(x) that Sj(T _1 x G C+. Hence cr _1 a; G -ff,. Since 
l(asj) < 1(a) we have j G D T (a). Hence cr _1 a: g" CdJct), and so i ^ &Cd t (o), a 
contradiction. □ 

2.3. Generating functions for monoconditional cones. Let V£ = V* <E)r C. 

Extend ( , ) to V x V^ by C-linearity in the second argument. Let r C V be 
a lattice and 5 G V. Suppose that there is a nonempty open subset B C V^ 
such that the series X^esnr e~^ x,ip ^ converges for <p £ B and has a meromorphic 
continuation to V^ . We denote this continuation by fs and call it the generating 
function of S with respect to T. 

Example 2.7. If C C V is a cone, let 

C v := {<p G V* | Vi G C : {x, ip) > 0} G F* 

be its dual cone. The complexified dual of C is defined as 

C% := {(p € V£ \ V x € C ; $t((x, <p)) >0} = C v +iV*. 

Let C C V be a salient cone, rational with respect to V. Then X^ecnr e 
converges on the interior of Cq and has a meromorphic continuation fc to V^. 
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From now on, suppose that W is crystallographic, i.e., that there is a W-invariant 
lattice r in 7. A full-dimensional simplicial cone C C V is called unimodular 
(with respect to T) if it is generated by a basis of T. These generators are called 
primitive. 

Theorem 2.8. In the situation of \Lemma 2.5\ suppose that C+ is unimodular with 
respect to T. Let &i, . . . , b n be the primitive generators of C+, enumerated in the 
unique way such that bj ^ Hj for j G {1, . . . , n — 1}. Then the generating function 
of C is 

n -(obj,ip) 



Mr) = E 



In practice, T is often endowed with a distinguished basis. In this case, it is often 
more convenient to work with the following formulation. 

Corollary 2.9. In the situation of \Theorem 2.8\. let ex,...,e n be a basis of T . 
Define coordinates Zj on V£ by Zj(tp) :— e^^ ej ^ . For a — a\e\ + • • ■ + a n e n G T, 
Then 



n, 



fc = E U <^M< T ' 

<7&W 



z 



(1 )...(! 



Proof of \Theorem 2.81 As C+ is unimodular with primitive generators bi, . . . ,b n , 
its generating function is 

fc +&) = (i_ e -<b 1 ,^)...(i_e-(6„, V )) ' 

With unimodularity it also follows that each generator of C+ is only one lattice 
hyperplane away from the opposite facet. Hence 

c b} n r = (c+ \ Hj) n r = (c+ n r) + bj 

for all j G {1, . . . , n - 1}. More generally, C j n T = (C+ n T) + J2jeJ b J for an y 
J G {1, — 1}. Applying this observation to J = D r (cr) for a a G W and 

rephrasing it in terms of generating functions, one obtains 

r / n _ LljeD T (cj) c 

JC DrM (P) - (i_ e -(bi^>) . . . (1 _ e -(bn,f)) ' 

Hence for all a G W it follows that 

FT , > e~( ab i<P) 

f / s _ lljeD r (a) K 

Uc DAa) W) - ^ _ e _( CT6li¥3 )) e -(a6„,VP>) ' 

By proposition 2"76| /c = J2aew fcC Dl{t7) , which proves the formula. □ 
3. Cones with the symmetry of a simplex 

IThcorcm 2.81 specializes to more concrete identities once we fix a particular al- 
most irreducible reflection group W . The case of W being the group of symmetries 
of a simplex has been treated in [2]. We include this case here to show how the 
result can be derived from IThcorcm 2 .81 

Let S n denote the group of permutations of the set {1, . . . , n}. For n G S n , we 
define the descent set of ir as 

(6) D(n) := {j £ {1, . . . , n — 1} | 7r(j) > n(j + 1)} . 
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This is the standard definition used in the literature on permutations. 

The group S n acts on R n by permutation of the components. For tt G S n , let 
<Ttc G 0(R") denote the transformation by which it acts on R™. Let W = {ov | 
7T G S n } C 0(R"). Then W is the group of symmetries of the (n — l)-dimensional 
standard simplex. For j = 1, . . . , n — 1, let Sj G W be the transposition of the jth 
and (j + l)st component in R™. Then si, . . . , s n -i are simple generators of W . 

The following shows that the definitions of descent given in ([2]) and (j6|) agree. 

Proposition 3.1 ( 4, Proposition 1.5.3]). D T (a 7V ) — D(ir) for all ir £ S n . 

Our main result in this section is the following. 

Proposition 3.2 ( 2, Theorem 1]). Fix integers a\ < ■ ■ ■ < a n such that a\ + • — h 
a n = 1. Let 

C := {x G R" | V7r e S n ■ ai^ w (i) H h a„i T ( n) > 0} . 

Lei Ej := ai + • • • + dj /or j G {1, . . . , n — 1}. T/ie generating function of C with 
respect to Z" is 

' ~ 1 - zi ■ ■ ■ Zn n;=i (i - • ■ • re =1 ' 

Note that the condition on the ai to be increasing is a normalization rather than 
a restriction. 



Proof. The cone C is symmetric and monoconditional for W . Let F — {x G 
R" xi > • • • > x n }, a fundamental domain for W. Then our chosen simple 
generators s%, . . . , s n ~i of W are the reflections at the facet hyperplanes of F. Let 
xq = (— ai, . . . , —a n ) G i* 1 . By the proof of ILemma 2.51 

C + = {x e F | (m, x ) <0} = {x e F \ aixi H h a n x n > 0} 

= {a; 6 R" | Ax > 0} , 



where 



.4 



/ 1 




• 



\ 





-1 



The determinant of A is ai + • • • + a n = 1, i.e., A is unimodular and so is C+. Let 
bi, . . . , b n be the primitive generators of C+, enumerated in the unique way such 
that bj Hj for j G {1, . . . , n— 1}. Then by |Corollary 2.9[ the generating function 
ofCis 



fc 



^ (1 



)...(! _^6») 
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proposition 3. 2| follows once we describe D r (a), bj, and the action of W explicitly. 
The inverse of A is 



( ^. 

-El 



B := A' 1 = 



Ei 



1 



\ 



e: 



-Ex 



n-1 

E n _l 1 



where E^ := 1 — Ej. Then 6j is the jth column vector of B. Let 

if j = n, 
Ej if i < j < n, 
'j if j < i 

be the ith component of bj, i.e., the (i,j)th component of -B. As defined above, 
with 7r 6 5 n we associate ov G O(n) by ove^ = ewj). Then W = {ov | 7r G £„} and 
we have D T (a- R ) — D(ir). Hence 



fc 



n 



\ - n 3 £A(^) z ' T7rbj 



y~v rijG-D(7r) rii=l Z 7r(i) 

1 x - rijecw^i 



(i) 



1-Z1 



• • • z - 4l n U ^)- E ^ nj=i 



□ 



4. Cones with hyperoctahedral symmetry 

We now consider the case of cones which are symmetric under the action of a 
hyperoctahedral group. Let W C O(n) be the hyperoctahedral group on the first 
n — 1 components of R™. Let si G be the sign change in the first component 
and, for j = 2, . . . , n — 1, let Sj 6 W be the transposition of the (j — l)st and jth 
component in R™. Then s%, . . . , s„_i are simple generators of W. 

For combinatorial (as opposed to geometric) arguments, it is often more con- 
venient to use the following parameterization of the hyperoctahedral group: For 
7r G SVi-i and e G {±1}™ _1 , define a v _ e G O(n) by 

where we use the convention that n(n) := n for n G S n —x and e n := 1 for s G 
{il}"" 1 . Then W = {ov, e | 7T G S„_i, e G {il}"" 1 }. Let B n -i denote the 
set SVi-i x {±1}" _1 , endowed with the group structure such that a : — > W 
becomes an isomorphism of groups. 
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In terms of this parameterization, the right descent set of W can be expressed 
more explicitly. For (ir, e) G B„-i let 

(8) D(n, e) := {j G {1, . . . , n - 1} | e^nr(j - 1) > Sj 7r(j)} 
with the convention that £o7r(0) := 0. Then the following holds. 
Proposition 4.1 ( 4, Proposition 8.1.2]). For all (n,e) G -B n -i, we have 

D t {a^ e ) = D(k,e) . 

Note that the descent set defined in (jHJ is translated by +1 with respect to 
definitions found in the literature on signed permutations. This is because to have 
a consistent setup in Isection 2l we always start the enumeration of the simple 
reflections with 1, whereas from a signed permutations perspective it is convenient 
to start this enumeration with 0. 

We define the descent statistic on the hyperoctahedral group by setting the 
descent number 

des(-7r, e) := \D(ir, e) 
for (tt, e) G -B n _i. Similarly, the major index is 

maj(7r,£):= ^ (j - 1) 

jeD(Tr,e) 

and the comajor index is 

comaj(7r,e) := (n-j) 

j£D(ir,e) 

for (7r,e) G B n _\. It follows that we have the relationship 

(9) comaj(7r, e) = (n — l)des(-7r, e) — maj(7r, e) . 

4.1. The multivariate generating function. In this situation, |Corollary 2.9| spe- 
cializes as follows. 

Proposition 4.2. Fix integers < a% < ■ ■ ■ < a n _i ^ 0. Let 
C := {x G R™ | Vvr G S n - U £ G {il}"" 1 : 

£iOiX ff (i) + • • • + £„_ia„_ix 7r („_ 1 ) < i n } . 
XTie generating function of C with respect to Z" is 

_ 1 ^ njeD(ir,e) Oi=j ^(i)^n* 

~~ 1 - Z ^ ^ rrn-1 A. TT"-! „£i r aA ' 

™ 7reS„-i £ e{±l}»-i = l ^! — Hi=j \(ifn ) 

Note that the condition on the to be nonnegative and increasing is a normal- 
ization rather than a restriction. 

Proof. The cone C is symmetric and monoconditional for W . Let 

F := {x G R" | < xi < ■ ■ ■ < x„_i} , 

a fundamental domain for W. The s±, . . . , s n _i defined previously are the simple 
generators of W corresponding to F. Let xo := {a\, . . . , a n -i, — 1) G F. By the 
proof of lLcmma 2.51 

C+ = {x G F | (x, x ) < 0} = {x G F | aixi H h a„_ix„_i < x„} 

= {x G R" | Ax > 0} , 
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where 



/ 1 



.4 



0\ 



V 



o ■ 



-1 1 ' 
• • — On-l 1 



The matrix A and hence C+ is unimodular. Let bx, . . . , b n be the primitive gener- 
ators of C+, enumerated in the unique way such that bj Hj for j < n. Then by 
|Corollary 2.9[ the generating function of C is 



n 



(l-z CTb i)---(l-;z CTb ™) 



The inverse of A is 



/ 1 



S := A' 1 = 



1 • 
Si 



•• 1 

1 



where E,- 



+ a n -i- Then bj is the jth column vector of B. Let 

if i < j, 

1 if j < i < n or i = j = n, 
j if j < i = n 



be the ith component of bj, i.e., the (i, j)th component of B. By proposition 4T] 
and using our notation introduced at the beginning of this section, 



fc 



\ - rijeD r (<T) ^ 



E 



n 



ie£> r (<r„, e ) 



(7r,e)6B„_i 



n?=i(i 



.2" 



E 



n 



nn e, 



(7T,e)£B 
1 



„_! n™=i (i - nr=i <*J 
E 



Ilje£)(7r,e) rii=j 



□ 



(Tr.ejes,,-! n"=i i 1 - nr=/ z ?(i) z « ) 

4.2. Hyperoctahedral Eulerian polynomials. In the remainder of lsection 41 we 
provide applications of [Proposition 43] with connections to permutation statistics 
and Ehrhart theory. Our first application is well known, going back to [7] and [llj ; 
the polyhedral perspective of the following identity was first established in [TT] , also 
using Ehrhart theory. 
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Corollary 4.3 ([7], [11]). The hyperoctahedral Eulerian polynomials are given by 

oo 

t des(-K,e) _ ^ _ t)« ^( 2 fc + l) n -h k . 

(7T,E)eB„_i fc=0 

Proof. Let 

p = f-i,ir _1 . 



be the (n — l)-dimensional hypercube. Our strategy to prove Corollary 4.3 is to 
compute the Ehrhart series 



Ehrp(t) t^^lfcpnz"- 1 ! -t k 



k>Q 



of P in two different ways and to conclude by comparing the results. 
On the one hand, note that the cone C over P, 

C = {x 6 R" \ yj < n : \xj \ < x n } , 

is the cone considered in [Proposition 4.2| for a% = ■ ■ ■ = a n _2 = 0, a„_i = 1, so by 
[Proposition 4.2| its generating function is 

fc = T—r 



Since Ehrp(i) is obtained by evaluating fc at Z\ = ■ ■ ■ = z n -\ = 1, z n = t, we 
obtain 



7r,e)eB„_i Ilf=l (1 l ) 



_ 1 ^dos(7r,e) 
(l-t) n ^ 

On the other hand, by definition 

Ehrp(t) =^2(2k + l) n - 1 t k . 

k>0 

Together, we obtain 



and Corollary 4.3 follows. □ 

4.3. The distribution of the comajor index. We show here how to derive the 
distribution of the comajor index; while this is likely well known, we could not find 
an explicit statement in the literature. For fc £ N and a variable t, let 

[k] t := l + t + t 2 + --- + t k - 1 and [fc] t ! := [l] t [2] t • • • [k] t ■ 

Corollary 4.4. The distribution of the comajor index on the hyperoctahedral group 
is given by 

^ f comaj(7r, E ) = (1 + i)"" 1 [ n - \] t \ . 
(ir,e)e-B„-i 
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Proof. Let 

P = {xe TV 1 - 1 | \ Xl \ + ■ ■ ■ + < 1} 

be the (n — l)-dimensional cross-polytope. Our strategy to prove [Corollary 4.4| is 
to compute the Ehrhart series 

Ehrp(t) :=^2\kPnZ n - 1 \-t k 

k>0 

of P in two different ways and to conclude by comparing the results. 
On the one hand, note that the cone C over P, 

C = {x e R n | |xi| + • • • + |x n -i| < Xn} , 

is the cone considered in |Proposition 4.2] for a\ = ■ ■ ■ = a n -\ = 1, so by |Proposition 4.~2] 
its generating function is 

, 1 v-^ T\jeD(TT,e) rii=j Z Tv\i) Zn 

fc = 7ZT7- 



1 Zn (w,e)£B„-i n"=l i 1 ~ H7=j Z l\i) Z n 



Since Ehrp(t) is obtained by evaluating fc at z\ = ■ ■ ■ = z n -i = L z n = t, we 
obtain 

Ehr-'"- x riiei3(7r, £ ) nr=7 t 



(7r, e )GB„-i n"=i (i - n™=/ i 

7r,e) 

(i-*)n^i(i-* n - j ) 



On the other hand, it is known 3, Theorem 2.7] that 

Ehrp(f) = a+*r- 1 , 
w (l-*)" 

Together, we obtain 

E( T ,e) £ B„_^ COmaj(l ' £) = + 

(i-*)n?i 1 (i-* n_i ) ( 1_ *) n ' 



so 



(7r : e)GS„_i v y 

(i+*) T, - i n?=i 1 (i-* n - j ') 



(i -t)"- 1 

1 4-Th— j 

(i+*r- l II i ^h- 

(i + *) n -Mn-i]*i. □ 
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Figure 2. The rational polytope P — {x <E R 3 | \x\\ 



F2 



F3 



max{|a;i|, |x2|. 
ICorollary 46} 



F3|} ^ !}• Its Ehrhart series can be computed by 



Remark 4.5. The distributions for the descent and comajor index statistics on B n 
arise from studying simple choices of the a, from the set of 0/1-vectors. It would be 
interesting to determine the structure of the multivariate generating functions (or 
their specializations) when other 0/1-vectors are used; due to the hyperoctahedral 
symmetry of our cones, this amounts to studying the case 

(ai,a 2 , . . . ,a„_i) = (0, . . . , 0, 1, . . . , 1) . 

The resulting cones interpolate naturally between cones over hypercubes and cones 
over crosspolytopes. While we were not able to treat this family of polytopes using 
the methods exposed in this article, the following section shows how to do so for a 
different interpolation. 



4.4. Almost constant coefficients. In this section, we show how to use |Proposition 4~2| 
to obtain a closed form expression for the Ehrhart series of a family of rational poly- 
topes interpolating between the hypercube and the cross polytope (considered in 
subsections 14.21 and 14. 31 respectively). These are the polytopes P such that the 
cone over P is of the form considered in [Proposition 4.2 such that 01 through a„_2 
coincide. An example of such a polytope is shown in Figure 2"] 



Corollary 4.6. Let b,c>0, not both 0. Let 

P = {xe R™ -1 I c-(\xx\ + ■■■ + |x n _i|) + 6-max{ F i|,...,|a: n _i|} < 1} . 
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Then the Ehrhart series of P is 

'[c] t (l + t c ) n - l /(l-t a ) n ifb = 0, 



Ehrp(t) = 



[Qt Ek>o ([* + l]t« + nkhf- 1 t bk ifb>l. 



Proof. In |Proposition 4.2( we set 01 = • • • = a n -2 = c and a n -i = c + b. Then 
Ehrp(t) = fc(t) '■= /c(l) • • • , the generating function of C evaluated at 

Zl : . . . , Z n _l — 1, z n 

For & > 0, the generating function fcif) becomes 

ETT fc(n-j)+b 
, , , , = llje£>(7r,e) fc 

(i-i)n^i(i-^ c( "^ )+6 ) 

E/-/.c\comaj(7T,e) /j.jAdes(7r,e) 

(i-t)n^(i-(i c K) 

For c > 1, if b = then /c(i) = /p(^ c )[c]t where P is the (n — l)-dimensional 
crosspolytope discussed in Isubsection 4.3[ this can be seen by direct computation 
using [Proposit ion 4.2| and is also a consequence of the fact that crosspolytopes are 
reflexive [21 Chapter 4]. Otherwise, to simplify, we make use of a result of Chow 
and Gessel eq. (26)] to compute the joint distribution of descent and comajor 
index over B n , namely 

a ,dos(7r,e) maj(7r,e) 

do) j> + 1], + [fc],rx fe = J — . 

fc>o llj=o^ x 1 ) 

Observe that for (n 7 e) £ B n , using ([9]), 

^comaj(7r,e) _ ^n\des(-7r,e) ^^\roaj(ir,e) 

Thus, substituting into (fT0|). we get 



a ,des(ir,e)^comaj(7r,e) _ (a;q") dcs ' 7r ' e - ) (l/g)" 1 ^ 1 -"'' 

(tt,e)GB„ (7r,e)eB„ 



- ztf"*) + l] 1/9 + [fc] 1/9 )"(xg") fe 

i=0 fc>0 
n 



1=0 fc>0 

To get the numerator of fc(t)> we set n = n— 1, x = t b and q = i c in the last line 
above and get: 

_ n?=o(i - ^ +b ) E fc>0 ([fc + + m^r- 1 ^ 

fc>o 
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4.5. Coefficients in arithmetic progression and lecture hall partitions. If 

we further generalize the results of the previous subsections to allow a,i to be a 
linear function of i, then fc(t) can be expressed in terms of lecture hall partitions. 

Lecture hall partitions, introduced by Bousquet-Melou and Eriksson [6], are 
elements of the set 

L n = {Xe Z n | o< ^ < 4f < ••• < 

The following relationship between statistics on lecture hall partitions and sta- 
tistics on signed permutations follows from work of Pensyl and Savage [10] : 

(11) Y x \^] q ^Wy^W = ^)eB q _^ UM 

xeL n nr=o(i-^"-^ 2((i+i)+ - +n) ) 

where 



n 



stati(A) = 

i=l 



, stat 2 (A) = ^2i 

i=l 

cobin(7r, e) = (J -\ h n) 

jeD(TT,e) 

for A e L n , (n, e) e B n , and \x] — inf ([x, oo) H Z). 

We can apply [Proposition 4.2[ with zi = • • • = z n _i = 1, z n = t and appropriate 
choices of a%, to establish a surprising connection between lecture hall partitions 
and type B symmetrically constrained cones. 

Corollary 4.7. Let d > 0, c > —2d, and b > 0, not all 0. Define a\, . . . , a n -i by 
di = 2di + c (i = 1, . . . , n — 2), a„_i = 2d(n — 1) + c + b . 

Then 

Aei„-i 

Proof. Observe that for 1 < i < n—2, substituting the values of Oj into |Proposition 4.2| 
with Z\ = ■ ■ ■ = z n -i = 1, z n = t, and then using (|lll) . we obtain 

TT *OjH ha„_i 

, , . 2-,(*-,e)eB„-i LljeD(w,e) 1 

JC[t) = — _! 



(i-*)n;=i(i-^ + - +a - 1 ) 

V FT f b+(n-j)c+2d(j+--- + (n-l)) 
Z^( 7 r,£)g_B„_ 1 lljg£)(7r,£) b 



(i - 1) nr=i (! - t( & + c (™- i )+2d('i+---+(«-i)) 

^(7T,£)eB„ 



E/j-c\comaj(7T,£) f^fr\dcs(7T,£) ^2d^cobin(7T,£) 
(7r,e)6S n _iV t ) \ L ) \ L I 



(1 - i) nrJl^ 1 - i( b + c (™-*)+ 2d (H--- + (n-l)) 

^c^comaj(7r,e) ^b^dcs(7r,e") ^2c^cobin(7r,£) 

(l - 1) Y^Zii 1 - t( 6 + c (™- j )+ 2d ( 4 +---+(™- 1 )) 

^6 [" 2n— 2 ] + cstat l (A)+d stat2(A) 



1-t 



1-t ^ 



1(5 
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As an example, let n — 3, a\ — 2 and <22 = 4. Then 

t(1) "I" «2XV(2) 



C = {x G R 3 I Vtt G S 2 , e G {±1} 2 : 2e 1 x n r 1) + te 2 x n{2 ) < x 3 } , 



and from [Proposition 4.2| 

t X3 = 



I + 3i 3 + 3i 6 + i 10 



^ (l-iXi-^Xi-t 4 ) 



= 1 + t + i 2 + t 3 + 5t 4 + 5< 5 + 9i 6 + 9t 7 + 13< 8 + 13< 9 + • 
On the other hand, checking the corollary, we have 

1 2 [4fl+4[^] 1 + 4i 4 + 4< 6 + 4t 8 + 8i 10 + 8t 12 + 8i 14 



1 -t ^ l-t 

= 1 + t + t 2 + t 3 + 5i 4 + 5t 5 + 9t 6 

+ m 7 + m 8 + m 9 + ■■■ . 

5. Cones with symmetry of type D 

In this section, we consider the case of monoconditional cones with symmetry 
given by a Coxeter group of type D. Unsurprisingly, much of the setup in this 
section is similar to the hyperoctahedral case; the most notable new feature is that 
we consider lattice point enumeration with respect to a sublattice of the standard 
integer lattice. 

Throughout this section, let W be the finite reflection group of type D„_i on 
the first n — 1 components of R n . Specifically, let Si G O(n) be the reflection at 
the hyperplane {x G R™ | x\ + x^ — 0}. For j = 2, . . . , n — 1, let Sj G O(n) be the 
transposition of the (j — l)st and jth component in R n . Then si, . . . , s n _i are the 
simple generators of W. 

We next describe D T (a) and the action of W explicitly. Let 

E n -! := {e G {±l} n - 1 I e x ■ ■ ■ e n -i = 1} . 

For 7r G S n -i and s G E n _i, define av i£ G O(n) by cr„ jS ei = £ie n u\ for i < n and 
d TiE e n = e n . Then W = {(r„ e \ tt G S n _i, e G Lv„_i}. We use the convention that 
7r(n) := n for 7r G S^i-i and £„ := 1 for e G E n _i. 
For 7r G S^-i and e G £7„-i let 

£>(7r,e) := {j'6{1 n-1} | ^-i7r(j - 1) > £j7t(j)} 

with the convention that £o7r(0) := — £2?r(2). 

Proposition 5.1 ( 4, Proposition 8.2.2]). For all ov^ G W we /icwe 

D r ( ( r 7ri£ )=£>(7r,£). 
For a proposition P, we use the symbol 



[PI := 



I if P is true, 
if P is false. 



Proposition 5.2. Fix integers a-y, . . . ,a n -x such that \ai\ < a<x < ••■ < a n _i 7^ 0. 
Lei 

C := {z G R n I Vtt G £ G B„_i : 

£lOl^7r(l) + ' ' ' + £n-l a n-l :z; ir(n-l) < ^n} • 
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Let 

r := {x e Z" | xi = ■ ■ ■ = x n _ 1 mod (2)} 
The generating function of C with respect to T is 



E E 



n, 



GD(7T,£) l"Sr(l) 



i+y>3] 



1 ' ci ,- n l r n 1 / , 



— ei —ax 

Z 7T(1) Z ™ 



b=2] 



where Z\,...,z n are the coordinates corresponding to the standard lattice Z" C R™ . 

Note that the conditions on the ai are normalizations rather than restrictions. 

Proof. The cone C is symmetric and monoconditional for W . Let 

F := {x e R" | |x x | < x 2 < ■ ■ ■ < x n -i} , 

a fundamental domain for W. Our simple generators s±, . . . , s n -i defined at the 
beginning of this section are the simple generators of W corresponding to the facets 
of F. Let xq := (ai, . . . , a„, —1) S F. By the proof of ILcmma 2.5[ 

(7+ = {x e F | (x, x ) < 0} = {x 6 F | aixi H h a„_ix n _i < x„} 

= {x e R" | Ax > 0} , 



where 



/ 1 



A := 



■ 

-ai 



-1 1 







'Ori-1 1 



The inverse of A is 



1/2 -1/2 
1/2 1/2 



A' 



J 

o\ 





1/2 1/2 1 ■ 

E!/2E^/2 S 3 



•■ 1 

S n _i 1 



where Ej := Oj + • • • + a n -i and S' 2 := £2 — °i- Hence the L-primitive generators 
of C+ are the column vectors b% , . . . , b n of the matrix 



/ 1 
1 



B := 



-1 







1 1 2 2 '0 

Si S 2 2X3 2S„_i 1 
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As det(.B) = 2 n - 2 = \Z n /T\ it follows that C+ is unimodular. Note that h,...,b n 
are enumerated in the unique way such that bj Hj for j < n, where Hj is the 
reflection hyperplane for the reflection Sj . Hence by |Corollary 2.9| the generating 
function of C is 



n, 



(1 



ycrbi 



Let 



-1 if i = 1, j = 2, 

1 if j < i < n or i = j = n, 

if i < j > 2, 

Si if « = n, j = 1, 

£2 if i = n, j = 2, 

2£j if i = 71, 2 < j < n 

be the ?th component of 6j, i.e., the (i, j)th component of £?. Thus 



m=i(i-^o 



- E E 

es„-i ee£„- 

E E 



n, 6 D 



n;=i(i-^ b o 

rijg£)(7r,E) IL=1 Z Tx{i) 



-z5„_! n™=i ( i - rnu ^ 
E E 



rijer>(7r,£) ( ^(i 1 ) 2 " Ql 



b=2] 



l+L/>3] 



7r£S„-i £6B„ 



n^iMi 



~~ 1 z ^( £ i 1 ) z ™ ai 



i+[./>C 



□ 
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